Method and system for correcting field of view using user terminal information upon playback of 360-degree image

ABSTRACT

A method and a system for correcting a field of view (FOV) using tilt information of a user terminal upon playback of a 360-degree image. The method includes requesting to load a 360-degree image, correcting at least one of a tilt of the mobile terminal and a sensing value associated with a gesture input of the mobile terminal, for determining an FOV of the 360-degree image, with respect to the 360-degree image, and playing back the 360-degree image by exposing an FOV depending on at least the corrected tilt or the corrected sensing value.

CROSS-REFERENCE TO RELATED APPLICATIONS

A claim for priority under 35 U.S.C. § 119 is made to Korean PatentApplication No. 10-2016-0059320 filed May 16, 2016, and No.10-2016-0087641 filed Jul. 11, 2016, in the Korean Intellectual PropertyOffice, the entire contents of each of which is hereby incorporated byreference.

BACKGROUND

Embodiments of the inventive concept described herein relate totechnologies for correcting a field of view (FOY) of a 360-degree image.

As interest in virtual reality (VR) is enhanced, interest in VR contentmay be also increased. For example, Korean Patent Laid-open PublicationNo. 10-2002-0078141 (published on Oct. 18, 2002) discloses a system andmethod for producing a VR video image to obtain an object video using aplurality of video cameras and displaying the object video on a viewerin real time over a network and a local path.

VR content may be produced or captured as a panorama image of a360-degree omnidirectional view point. A user may perform stereoscopicviewing of the VR content, which is not provided from a conventionalvideo, and may select a direction he or she wants in all 360-degreedirections and may view the VR content, rather than a form of viewingdependent on a specific view point of a photographer.

The content itself may be provided as an image of a 360-degreeomnidirectional view point, but only a limited area may be consumedaccording to characteristics of a medium which consumes content, forexample, a VR player, a mobile terminal, a web, or the like.

Particularly, if the user consumes a 360-degree image in a mobileenvironment rather than a dedicated terminal, he or she may miss animportant scene or a specific scene depending on an FOV of content.

SUMMARY

Embodiments of the inventive concept provide a method and system forcorrecting an FOV of content using tilt information and/or gestureinformation of a mobile terminal upon playback of a 360-degree image.

According to an aspect of an embodiment, a field of view (FOY)correction method performed in a mobile terminal implemented with acomputer may include requesting to load a 360-degree image, correctingat least one of a tilt of the mobile terminal and a sensing valueassociated with a gesture input of the mobile terminal, for determiningan FOV of the 360-degree image, with respect to the 360-degree image,and playing back the 360-degree image by exposing an FOV depending on atleast the corrected tilt or the corrected sensing value.

The correcting of the at least one may include averaging tilt valuescollected by the mobile terminal and correcting the tilt of the mobileterminal relative to the averaged tilt value.

The method may further include, if the 360-degree image is included on atimeline of an application loaded into the mobile terminal or on awebpage, recording a tilt value detected by the mobile terminal. Thecorrecting of the at least one may include averaging the recorded tiltvalues and correcting the tilt of the mobile terminal relative to theaveraged tilt value.

The correcting of the at least one may include averaging tilt valuesrecorded in a cache on an application loaded into the mobile terminal orin a cookie or cache on a browser and correcting the tilt of the mobileterminal relative to the averaged tilt value.

The correcting of the at least one may include correcting the tilt ofthe mobile terminal relative to an average tilt value of the mobileterminal, the average tilt value being stored on a server which providesthe 360-degree image.

The correcting of the at least one may include averaging sensing valuesdepending on a gesture input which occurs on the mobile terminal andcorrecting sensing sensitivity of the gesture input relative to theaveraged sensing value.

The method may further include, if the 360-degree image is included on atimeline of an application loaded into the mobile terminal or on awebpage, recording a sensing value depending on a gesture input whichoccurs on the mobile terminal. The correcting of the at least one mayinclude averaging the recorded sensing values and correcting sensingsensitivity of the gesture input relative to the averaged sensing value.

The playing back of the 360-degree image may include executing automaticplayback by moving a current FOV to a basic FOV and exposing the currentFOV if an automatic playback function is executed for the played-back360-degree image.

The executing of the automatic playback may include moving the currentFOV of the 360-degree image to the basic FOV corresponding to initialviewing angle information set in the 360-degree image.

The executing of the automatic playback may include extracting a basicFOV based on pixel information or meta information of the 360-degreeimage and moving the current FOV of the 360-degree image to theextracted basic FOV.

According to another aspect of an embodiment, an FOV correction methodperformed in a server implemented with a computer may include receivinga protocol or a message for requesting a streaming service for a360-degree image from a user terminal, analyzing application informationincluded in the protocol or the message for requesting the streamingservice and determining a content consumption environment of the userterminal, if the content consumption environment is a mobileenvironment, connecting configuration information, to support thecorrection of at least one of a tilt of a mobile terminal and a sensingvalue associated with a gesture input of the mobile terminal, to the360-degree image, and transmitting the 360-degree image connected to theconfiguration information to the mobile terminal, the 360-degree imagebeing as a response to the protocol or the message for requesting thestreaming service.

The mobile terminal may play back the 360-degree image by correcting atleast one of the tilt of the mobile terminal and the sensing valueassociated with the gesture input, for determining an FOV of the360-degree image depending on the response and exposing an FOV dependingon at least the corrected tilt or the corrected sensing value.

The method may further include setting initial viewing angle informationwhich is a basic FOV with respect to the 360-degree image.

The setting of the initial viewing angle information may include settingthe initial viewing angle information using a center area based on pixelinformation of the 360-degree image.

According to another aspect of an embodiment, a computer program storedin a computer-readable medium may be used to execute an FOV correctionmethod performed in a mobile terminal implemented with a computer, themethod including requesting to load a 360-degree image, correcting atleast one of a tilt of the mobile terminal and a sensing valueassociated with a gesture input of the mobile terminal, for determiningan FOV of the 360-degree image, with respect to the 360-degree image,and playing back the 360-degree image by exposing an FOV depending on atleast the corrected tilt or the corrected sensing value.

According to another aspect of an embodiment, a mobile terminalimplemented with a computer may include at least one processorconfigured to execute a computer-readable instruction. The at least oneprocessor may be configured to request to load a 360-degree image,correct at least one of a tilt of the mobile terminal and a sensingvalue associated with a gesture input of the mobile terminal, fordetermining an FOV of the 360-degree image, with respect to the360-degree image, and play back the 360-degree image by exposing an FOVdepending on at least the corrected tilt or the corrected sensing value.

According to another aspect of an embodiment, a server implemented witha computer may include at least one processor configured to execute acomputer-readable instruction. The at least one processor may beconfigured to receive a protocol or a message for requesting a streamingservice for a 360-degree image from a user terminal, analyze applicationinformation included in the protocol or the message for requesting thestreaming service and determine a content consumption environment of theuser terminal. If the content consumption environment is a mobileenvironment, the at least one processor may connect configurationinformation, for supporting the correction of at least one of a tilt ofa mobile terminal and a sensing value associated with a gesture input ofthe mobile terminal, to the 360-degree image, and transmit the360-degree image connected to the configuration information to themobile terminal, the 360-degree image being as a response to theprotocol or the message for requesting the streaming service. The mobileterminal may play back the 360-degree image by correcting at least oneof the tilt of the mobile terminal and the sensing value associated withthe gesture input, for determining an FOV of the 360-degree image,depending on the response and exposing an FOV depending on at least thecorrected tilt or the corrected sensing value.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and features will become apparent from thefollowing description with reference to the following figures, whereinlike reference numerals refer to like parts throughout the variousfigures unless otherwise specified, and wherein:

FIG. 1 is a drawing illustrating an example of a network environmentaccording to an embodiment of the present disclosure;

FIG. 2 is a block diagram illustrating an internal configuration of anelectronic device and an internal configuration of a server in anembodiment of the present disclosure;

FIGS. 3 and 4 are drawings illustrating a characteristic of a 360-degreeimage;

FIGS. 5, 6, and 7 are drawings illustrating an FOV of a 360-degree imagedepending on a tilt of a mobile terminal;

FIG. 8 is a block diagram illustrating an example of elements which maybe included in a processor of a mobile terminal according to anembodiment of the present disclosure;

FIG. 9 is a flowchart illustrating an FOV correction method which may beperformed by a mobile terminal according to an embodiment of the presentdisclosure; and

FIG. 10 is a flowchart illustrating an example of a method which may beperformed by a server according to an embodiment of the presentdisclosure.

DETAILED DESCRIPTION

Hereinafter, a description will be given in detail of an embodiment ofthe inventive concept with reference to the accompanying drawings.

FIG. 1 is a drawing illustrating an example of a network environmentaccording to an embodiment. The network environment of FIG. 1 is anexample including a plurality of electronic devices 110, 120, 130, and140, a plurality of servers 150 and 160, and a network 170. FIG. 1 isonly an example for describing the inventive concept, and the number ofelectronic devices or the number of servers is not limited to FIG. 1.

Each of the plurality of electronic devices 110, 120, 130, and 140 maybe a fixed terminal or a mobile terminal, implemented with a computerdevice. Each of the plurality of electronic devices 110, 120, 130, and140 may be, for example, a smartphone, a mobile phone, a display devicesuch as a television (TV), a virtual reality (VR) player, a head mounteddisplay (HMD), a navigation device, a computer, a laptop, a digitalbroadcast terminal, personal digital assistants (PDA), a portablemultimedia player (PMP), a tablet personal computer (PC), or the like.For example, the electronic device 110 may communicate with the otherelectronic devices 120, 130, and 140 and/or the servers 150 and 160 overthe network 170 using a wireless or wired communication mode.

The communication mode is not limited thereto. The communication modemay include a communication mode using short-range wirelesscommunication as well as a communication mode using a communicationnetwork (e.g., a mobile communication network, a wired internet, awireless internet, a broadcast network, or the like) which may beincluded in the network 170. For example, the network 170 may includeone or more of networks such as a personal area network (PAN), a localarea network (LAN), a campus area network (CAN), a metropolitan areanetwork (MAN), a wide area network (WAN), a broadband network (BBN), acontent delivery network (CDN), and the Internet. Further, the network170 may include, but is not limited to, one or more of networktopologies including a bus network, a star network, a ring network, amesh network, a star-bus network, a tree or hierarchical network, andthe like.

Each of the servers 150 and 160 may be implemented with a computerdevice or a plurality of computer devices which communicate with theplurality of electronic devices 110, 120, 130, and 140 over the network170 and provide an instruction, a code, a file, content, a service orthe like.

For example, the server 160 may provide a file for installing anapplication to the electronic device 110 accessed over the network 170.In this case, the electronic device 110 may install the applicationusing the file provided from the server 160. Further, the electronicdevice 110 may access the server 150 based on control of an operatingsystem (OS) or at least one program (e.g., a browser or the installedapplication) included in the electronic device 110 and may receive aservice or content provided from the server 150. For example, if theelectronic device 110 transmits a service request message to the server150 over the network 170 based on control of the application, the server150 may transmit a code corresponding to the service request message tothe electronic device 110. The electronic device 110 may provide contentto a user by configuring and displaying a screen depending on a codebased on control of the application.

FIG. 2 is a block diagram illustrating an internal configuration of anelectronic device and an internal configuration of a server in anembodiment of the present disclosure. FIG. 2 illustrates an internalconfiguration of an electronic device 110 as an example of oneelectronic device and an internal configuration of a server 150 as anexample of one server. The other electronic devices 120, 130, and 140 orthe server 160 may have the same or different internal configurationfrom the electronic device 110 or the server 150.

The electronic device 110 may include a memory 211, a processor 212, acommunication module 213, and an input/output interface 214. The server150 may include a memory 221, a processor 222, a communication module223, and an input/output interface 224. Each of the memories 211 and 221may be a computer-readable storage medium and may include a randomaccess memory (RAM) and a permanent mass storage device such as a readonly memory (ROM) or a disc drive. Further, each of the memories 211 and221 may store an operating system (OS) or at least one program code(e.g., a code for a browser, an application, or the like installed andexecuted in the electronic device 110). Such software elements may beloaded from a separate computer-readable storage medium independent ofthe memories 211 and 221. Such a separate computer-readable storagemedium may include a computer-readable storage medium such as a floppydrive, a disc, a tape, a digital versatile disc/compact disc-ROM(DVD/CD-ROM), a memory card, or the like. In another embodiment, thesoftware elements may be loaded into the memories 211 and 221 via thecommunication module 213 and 223 rather than a computer-readable storagemedium. For example, at least one program may be loaded into thememories 211 and 221 based on a program (e.g., the above-mentionedapplication) installed by files provided over the network 170 fromdevelopers or a file distribution system which distributes a file forinstalling an application.

Each of the processors 212 and 222 may be configured to processinstructions of a computer program by performing basic calculation andlogic and input/output arithmetic operations. The instructions may beprovided to the processor 212 and 222 by the memories 211 and 221 andthe communication modules 213 and 223. For example, each of theprocessors 212 and 222 may be configured to execute instructionsreceived according to a program code stored in a storage device such asthe memories 211 and 221.

The communication modules 213 and 223 may provide a function such thatthe electronic device 110 and the server 150 communicate with each otherover the network 170 and may provide a function for communicating withanother electronic device (e.g., the electronic device 120) and anotherserver (e.g., a server 160). For example, a request generated accordingto a program code stored in a storage device such as the memory 211 bythe processor 212 of the electronic device 110 (e.g., a request for apanorama image service) may be transmitted to the server 150 over thenetwork 170 based on control of the communication module 213. Incontrast, a control signal, an instruction, content, a file, or the likeprovided based on control of the processor 222 of the server 150 may bereceived at the electronic device 110 via the communication module 213of the electronic device 110. For example, a control signal, aninstruction, or the like of the server 150, received via thecommunication module 213, may be transmitted to the processor 212 or thememory 211, and content, a file, or the like of the server 150 may bestored in a storage medium which may be further included in theelectronic device 110.

The input/output interface 214 may be a means for an interface with aninput/output device 215. For example, an input device included in theinput/output device 215 may include a device such as a keyboard or amouse, and an output device included in the input/output device 215 mayinclude a device such as a display for display a communication sessionof an application. For another example, the input/output interface 214may be a means for an interface with a device, such as, a touch screen,in which functions for an input and an output are integrated into onefunction. As a more detailed example, when the processor 212 of theelectronic device 110 processes an instruction of a computer programloaded into the memory 211, a service screen or content configured usingdata provided from the server 150 or the electronic device 120 may bedisplayed on a display via the input/output interface 214.

Further, in other embodiments, the electronic device 110 and the server150 may include more elements than the elements of FIG. 2. However,there is no need for clearly illustrating most conventional elements.For example, the electronic device 110 may be implemented to include atleast part of the input/output device 215 or may further include otherelements such as a transceiver, a global positioning system (GPS), acamera, various sensors, and a database. As a more detailed example, ifthe electronic device 110 is a smartphone, it may be seen that theelectronic device 110 may be implemented to further include a variety ofelements such as an accelerometer or a magnetic sensor, a gyro sensor, acamera, various physical buttons, a button using a touch panel, aninput/output port, and a vibrator for vibration, generally included inthe smartphone.

Embodiments relate to technologies for correcting an FOV of a 360-degreeimage and may be applied to a service field for providing an imageservice, a broadcast service, and a service using a 360-degree imagesuch as a virtual experience environment, a device field for supportingthe playback of a 360-degree image, or the like.

Embodiments including examples disclosed in detail in the specificationmay correct a playback focus of a 360-degree image due to a differencein a tilt or a gesture input in a mobile playback environment, thusaccomplishing considerable advantages in view of efficiency,convenience, cost saving, or the like.

A 360-degree image may be described as a representative example in thespecification. However, embodiments are not limited thereto. Forexample, the specification may be described for all panorama images,such as spherical content and a three-dimensional (3D) image, a viewpoint (or a view angle) of which may be adjusted, other than the360-degree image.

For example, the 360-degree image may be produced using a 360-degreecamera or the like. For example, as shown in FIG. 3, a 360-degree camera3 may obtain an image 30 of a 360-degree omnidirectional view point bysimultaneously capturing an image using several lenses which may capturethe image at a view angle of 360 degrees and stitching the capturedphotos/images.

In the case of the 360-degree image, content itself may be provided asan image of 360-degree omnidirectional view point, but only a limitedregion of the content may be consumed according to a characteristic of amedium which consumes the content, for example, a VR player, a mobileterminal, a web, or the like. Referring to FIG. 4, a 360-degree image 40may be spherical content having a 360-degree omnidirectional view point,but there may be a missing area 43 in all areas of transverse andlongitudinal directions except for an FOV 41 which is a range (e.g., aplayer view angle) which is actually exposed on a video area viewed by auser. A focus on an area of the longitudinal direction may be a biggerproblem than an area of the transverse direction in a contentconsumption environment.

If a 360-degree image is consumed in a mobile environment, a tilt of amobile terminal may be sensed using a sensor (e.g., an accelerometer, amagnetic sensor, or the like) included in the mobile terminal. An imageportion of a viewing angle corresponding to the tilt of the mobileterminal among 360-degree omnidirectional view points of the 360-degreeimage may be played back.

FIGS. 5 and 6 are examples of an initial content loading screen and aredrawings illustrating an FOV of a 360-degree image depending on a tiltof a mobile terminal.

FIG. 5 illustrates a general terminal tilt 51 and an FOV 500 dependingon the tilt 51 upon use of the mobile terminal. Since a horizontal angleof an image capture camera at a time when a 360-degree image is produceddiffers in tilt from a basic angle of a player upon viewing using amobile terminal, there may be a problem of missing a specific importantscene upon loading of initial content without efficiently exposing ascene intended by a content producer or a photographer on an initialscreen.

Since a tilt value of a mobile terminal of a general user is notcorrected in a mobile playback environment, the user must adjust a tiltof the mobile terminal to focus directly on an important scene of a360-degree image. As shown in FIG. 6, if a tilt 63 of the mobileterminal is adjusted to form an angle of 90 degrees with the ground, anFOV 600 corresponding to the tilt 63 may be exposed on a screen of themobile terminal.

In other words, since the 360-degree image is a panorama image having anomnidirectional viewing angle, only a partial area may be exposed on ascreen depending on a viewing angle of a playback view point. As shownin FIG. 7, since only a partial area II is exposed depending on a tiltof a mobile terminal in the entire vertical area I of the same playbackview point in a mobile playback environment, a difference of informationexposed on a screen of the mobile terminal may occur according to theFOV II.

When an important scene intended by a content producer or a photographeris the portion II where the information ‘ABC’ is exposed, if a tilt of amobile terminal does not correspond to the area II, the user may missthe important scene.

If a tilt for focusing on an important scene of a 360-degree image isnot a general terminal tilt of a mobile terminal, because the user viewsthe 360-degree image at an uncomfortable angle or position, the userwill be inconvenienced or burdened.

An image portion of a viewing angle corresponding to a gesture input ofthe user may be played back based on the gesture input (e.g., a draginput or the like) which occurs on a 360-degree image as well as a tiltof a mobile terminal in a mobile playback environment. A difference ininformation exposed on a screen of the mobile terminal may occuraccording to an FOV corresponding to a gesture input by the user as wellas the tilt of the mobile terminal.

Therefore, embodiments may provide a function of correcting a focus ofcontent playback due to a difference in a tilt or gesture input in amobile playback environment. The inventive concept may provide anenvironment of focusing on a 360-degree image and consuming the focused360-degree image by correcting a tilt value of a mobile terminal and asensing value associated with a gesture input of the mobile terminalupon loading of content in a mobile playback environment.

FIG. 8 is a block diagram illustrating an example of elements which maybe included in a processor of a mobile terminal according to anembodiment. FIG. 9 is a flowchart illustrating an FOV correction methodwhich may be performed by a mobile terminal according to an embodiment.

As shown in FIG. 8, the mobile terminal may correspond to an electronicdevice 110 described with reference to FIGS. 1 and 2. A processor 212 ofthe mobile terminal may include a recording unit 810, a correction unit820, and a content playback unit 830 as elements. Such elements of theprocessor 212 may be expressions of different functions performed by theprocessor 212 based on a control command provided by at least oneprogram code. For example, the recording unit 810 may be used as afunctional expression that the processor 212 operates to record a tiltvalue of the mobile terminal and/or a sensing value of a gesture inputof the mobile terminal.

The processor 212 and the elements of the processor 212 may performoperations S910 to S940 included in the FOV correction method of FIG. 9.For example, the processor 212 and the elements of the processor 212 maybe implemented to execute an instruction according to a code of anoperating system (OS) included in a memory 211 of FIG. 2 and the atleast one program code. Herein, the at least one program code maycorrespond to a code of a program implemented to process the FOVcorrection method.

The FOV correction method may fail to be performed in a shown order.Some of the operations of the FOV correction method may be omitted fromthe FOV correction method, or an additional operation may be furtherincluded in the FOV correction method.

In operation S910, the processor 212 may load a program code stored in aprogram file for the FOV correction method into the memory 211. Forexample, the program file for the FOV correction method may be store ina permanent storage device. The processor 212 may control the mobileterminal to load the program code into the memory 211 from the programfile stored in the permanent storage device via a bus.

In this case, the processor 212 and each of the recording unit 810, thecorrection unit 820, and the content playback unit 830 included in theprocessor 212 may be different functional expressions of the processor212 for executing an instruction of a corresponding portion in theprogram code loaded into the memory 211 and executing subsequentoperations S920 to S940. To execute operations S920 to S940, theprocessor 212 and the elements of the processor 212 may directly processan arithmetic operation or may control the mobile terminal.

In operation S920, the recording unit 810 may record at least one of atilt value of the mobile terminal and a sensing value depending on agesture input of the mobile terminal when an application or a webpageincluding a 360-degree image is used. In general, the mobile terminalmay provide 3-axis tilt information using a sensor included in themobile terminal, for example, an accelerometer or a magnetic sensor, andmay provide a sensing value depending on a gesture input using a touchsensor and a motion sensor. For example, the recording unit 810 mayrecognize whether there is a 360-degree image on a timeline of anapplication or on a webpage in a process of loading the mobileapplication or the webpage. If there is a 360-degree image on thetimeline of a mobile application or on a webpage, the recording unit 810may record a tilt value detected by the mobile terminal and/or a sensingvalue of a gesture input detected by the mobile terminal while thecontent is used. There may be various methods of recognizing a360-degree image in the mobile terminal. For example, a content providermay mark whether to include a 360-degree image using a tag or metadata,and the mobile terminal may recognize the 360-degree image through themark. For another example, if a 360-degree image is included, thecontent provider may provide a JavaScript or an application programminginterface (API) for requesting a tilt value and/or a sensing value of agesture input to the mobile terminal. Further, it is possible to providean average tilt value of the user or other users and/or an averagesensing value for gesture inputs on a script. For another example, amobile browser or an application may verify an information tag ormetadata in a document object model (DOM) parsing process or may call anassociated JavaScript API, and thus, the mobile terminal may recognize a360-degree image. For another example, the mobile terminal may provide asensor API for recognizing a 360-degree image itself. Thus, therecording unit 810 may recognize an environment in which a 360-degreeimage is used through various methods and may record a tilt valuedetected by the mobile terminal and/or a sensing value depending on agesture input detected by the mobile terminal while the 360-degree isused.

In operation S930, the correction unit 820 may correct at least one of atilt of the mobile terminal and a sensing value associated with agesture input of the mobile terminal at a time when loading of a new360-degree image is requested. In this case, the correction unit 820 mayaverage previously recorded tilt values and may correct a reference tiltof the mobile terminal for the content relative to the averaged tiltvalue. The correction unit 820 may correct a horizontal value (a basicangle value) for a 360-degree image relative to tilt information of themobile terminal which is being used by a user at a current time. If tiltvalues of the mobile terminal are not sufficiently collected or if thereis no tilt value of the mobile terminal at a content loading time, thecorrection unit 820 may use a terminal tilt value stored in a cache ofan application or in a cookie or cache on a mobile browser. For anotherexample, the correction unit 820 may use a terminal tilt average valueof the user, stored in a server 150 of FIG. 2 which provides a360-degree image. In other words, the correction unit 820 may averagetilt values of the mobile terminal depending on user characteristicsbefore a 360-degree image is played back and may correct a referencetilt of the mobile terminal using the averaged tilt value. Thecorrection unit 820 may record a sensing value depending on a gestureinput previously generated in the mobile terminal and may average therecorded sensing values, thus correcting sensing sensitivity of agesture input of the mobile terminal for the content relative to theaveraged sensing value. In other words, the correction unit 820 maycorrect a gesture input, such as a drag input of the user, which is oneof elements for determining an FOV of a 360-degree image, before the360-degree image is played back.

In operation S940, the content playback unit 830 may play back contentdepending on at least the corrected tilt or the corrected sensing valueof the mobile terminal for the 360-degree image requested to be loaded.The correction unit 820 may correct a basic tilt for the 360-degreeand/or a sensing value associated with a gesture input using an averagetilt frequently used by the user when the 360-degree image is initiallyloaded. In this case, the content playback unit 830 may determine andexpose an initial FOV of the 360-degree image based on the correctedtilt and/or the corrected sensing value of the gesture input. Therefore,the inventive concept may minimize inconvenience for the user to use a360-degree image by correcting a tilt value and/or a gesture input of amobile terminal for the 360-degree image relative to a tilt and/or agesture input of the mobile terminal usually and frequently used by theuser when the 360-degree is loaded and playing back the contentdepending on the corrected tilt and/or sensing value associated with thegesture input.

If the user directly operates the mobile terminal using a tilt or agesture, such as a drag, of the mobile terminal and moves an FOV of a360-degree image to view a desired scene, it may be difficult to move toan initial value (a basic FOV) in view of a content producer or aphotographer. Therefore, the inventive concept may provide an automaticplayback function for automatically returning to an initial value of a360-degree image which is being currently played back.

The content playback unit 830 may perform automatic playback to a basicFOV by moving and exposing a current FOV to the basic FOV of a360-degree image when the automatic playback function is executed forthe 360-degree image which is being played back according to at leastthe corrected tilt or the corrected sensing value of the gesture input.For example, the content playback unit 830 may move a current FOV of a360-degree image to a basic FOV corresponding to initial viewing angleinformation previously set in the 360-degree image and may performautomatic playback. For another example, the content playback unit 830may extract a center area of a view point based on pixel information (or3-axis information) or meta information of a 360-degree image as a basicFOV and may move a current FOV of the 360-degree image to the extractedbasic FOV, thus performing automatic playback. If the automatic playbackfunction is executed, the content playback unit 830 may initialize tiltinformation of the mobile terminal or a sensing value associated with agesture input of the mobile terminal and may synchronize terminalinformation to screen playback changed due to automatic playback.

Thus, while a 360-degree image is consumed at a view point the userwants based on a tilt or gesture input of the mobile terminal, an FOVarea of the 360-degree image may easily and conveniently return to aninitial value always in view of a content producer or a photographerthrough the automatic playback function.

FIG. 10 is a flowchart illustrating an example of a method which may beperformed by a server according to an embodiment.

In operation S1010, a server 150 of FIG. 2 may receive a separateprotocol or message (e.g., a hypertext transfer protocol (HTTP) or thelike) for requesting a streaming service for a 360-degree image from anelectronic device 110 of FIG. 2. In this case, at least one of a tiltvalue of the electronic device 110 or a sensing value of a gesture inputof the electronic device 110 may be included on the service requestprotocol or message. The service request protocol or message may includeapplication information and the like for requesting a streaming service.

In operation S1020, the server 150 may verify the applicationinformation included in the service request protocol or message. If anHTTP request is received in a mobile environment, the server 150 mayanalyze request information received from the mobile terminal and maydetermine a content consumption environment.

If an environment in which the 360-degree image will be consumed is anenvironment using the mobile terminal, in operation S1030, the server150 may connect application configuration information to support thecorrection of at least one of a tilt of the mobile terminal and asensing value associated with a gesture input of the mobile terminal.The server 150 may set initial viewing angle information of contentthrough a preprocessing procedure of the 360-degree image or may setinitial viewing information by inserting a sensing value of anapplication. If there is no input of a separate sensing value, theserver 150 may set a center area using an initial viewing angle based onpixel information in a vertical area of content. Thus, if loading of a360-degree image is requested in a mobile environment, the server 150may connect information, for correcting at least one of a reference tiltof a mobile terminal and a sensing value associated with a gesture inputof the mobile terminal, to content.

In operation S1040, the server 150 may transmit a response correspondingto the service request protocol or message to the electronic device 110.In other words, if loading of a 360-degree image is requested in amobile environment, the server 150 may respond to setting information,to support the correction of at least one of a tilt of the mobileterminal and a sensing value associated with a gesture input of themobile terminal, together with the content.

Therefore, when providing a streaming service for a 360-degree image tothe mobile terminal, the server 150 may provide an environment forcorrecting a basic tilt value and/or a sensing value associated with agesture input of the mobile terminal, for determining an initial FOV ofthe 360-degree image.

As such, embodiments may increase content consumption and may enhanceuser satisfaction by correcting an FOV of content using tilt informationand/or a gesture input of a mobile terminal when a 360-degree image isplayed back and focusing on and consuming the content in a mobileenvironment. Particularly, embodiments may prevent the user from missingan important scene and may effectively save the purpose of productionincluded in content by correcting a tilt value of the mobile terminaland/or a sensing value associated with a gesture input of the mobileterminal upon loading of initial content in a mobile playbackenvironment and exposing a scene intended by a content producer or aphotographer on an initial screen.

The foregoing devices may be realized by hardware elements, softwareelements and/or combinations thereof. For example, the devices andcomponents illustrated in the exemplary embodiments of the inventiveconcept may be implemented in one or more general-use computers orspecial-purpose computers, such as a processor, a controller, anarithmetic logic unit (ALU), a digital signal processor, amicrocomputer, a field programmable array (FPA), a programmable logicunit (PLU), a microprocessor or any device which may executeinstructions and respond. A processing unit may implement an operatingsystem (OS) or one or more software applications running on the OS.Further, the processing unit may access, store, manipulate, process andgenerate data in response to execution of software. It will beunderstood by those skilled in the art that although a single processingunit may be illustrated for convenience of understanding, the processingunit may include a plurality of processing elements and/or a pluralityof types of processing elements. For example, the processing unit mayinclude a plurality of processors or one processor and one controller.Also, the processing unit may have a different processing configuration,such as a parallel processor.

Software may include computer programs, codes, instructions or one ormore combinations thereof and may configure a processing unit to operatein a desired manner or may independently or collectively control theprocessing unit. Software and/or data may be permanently or temporarilyembodied in any type of machine, components, physical equipment, virtualequipment, computer storage media or units or transmitted signal wavesso as to be interpreted by the processing unit or to provideinstructions or data to the processing unit. Software may be dispersedthroughout computer systems connected via networks and may be stored orexecuted in a dispersion manner Software and data may be recorded in oneor more computer-readable storage media.

The methods according to the above-described exemplary embodiments ofthe inventive concept may be implemented with program instructions whichmay be executed through various computer means and may be recorded incomputer-readable media. The media may also include, alone or incombination with the program instructions, data files, data structures,and the like. The program instructions recorded in the media may bedesigned and configured specially for the exemplary embodiments of theinventive concept or be known and available to those skilled in computersoftware. Computer-readable media include magnetic media such as harddisks, floppy disks, and magnetic tape; optical media such as compactdisc-read only memory (CD-ROM) disks and digital versatile discs (DVDs);magneto-optical media such as floptical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory (ROM), random access memory (RAM), flash memory, andthe like. Program instructions include both machine codes, such asproduced by a compiler, and higher level codes that may be executed bythe computer using an interpreter. The described hardware devices may beconfigured to act as one or more software modules to perform theoperations of the above-described exemplary embodiments of the inventiveconcept, or vice versa.

Embodiments may increase content consumption and may enhance usersatisfaction by correcting an FOV of content using tilt informationand/or a gesture input of a mobile terminal when a 360-degree image isplayed back and by focusing on and consuming the content in a mobileenvironment.

Embodiments may prevent the user from missing an important scene and mayeffectively save the purpose of production included in content bycorrecting a tilt value and/or a gesture input of a mobile terminal uponloading of initial content in a mobile playback environment and exposinga scene intended by a content producer or a photographer on an initialscreen.

While a few exemplary embodiments have been shown and described withreference to the accompanying drawings, it will be apparent to thoseskilled in the art that various modifications and variations can be madefrom the foregoing descriptions. For example, adequate effects may beachieved even if the foregoing processes and methods are carried out indifferent order than described above, and/or the aforementionedelements, such as systems, structures, devices, or circuits, arecombined or coupled in different forms and modes than as described aboveor be substituted or switched with other components or equivalents.

Therefore, other implements, other embodiments, and equivalents toclaims are within the scope of the following claims.

What is claimed is:
 1. A field of view (FOV) correction method performed in a mobile terminal implemented with a computer, the method comprising: requesting to load a 360-degree image; correcting a basic tilt of the mobile terminal to a corrected tilt, and using the corrected tilt to determine an FOV of the 360-degree image; and playing the 360-degree image by exposing the FOV, wherein the FOV is different than an FOV that would be exposed using the basic tilt; wherein said correcting corrects a horizontal value of the FOV; and wherein the FOV determined using the corrected tilt includes an important scene.
 2. The method of claim 1, wherein the basic tilt is calculated by averaging tilt values of the mobile terminal.
 3. The method of claim 1, wherein the tilt values that are averaged are recorded in a cache on an application loaded into the mobile terminal or in a cookie or cache on a browser.
 4. The method of claim 1, wherein the basic tilt is an average tilt value of the mobile terminal that is stored on a service which provides the 360-degree image.
 5. The method of claim 1, wherein the corrected tilt forms an angle of approximately 90 degrees with the ground.
 6. The method of claim 1, wherein the mobile terminal comprises a smartphone comprising one or more sensors, the method further comprising measuring the the basic tilt using said one or more sensors.
 7. The method of claim 1, wherein the corrected tilt is based on one or more attributes of the 360-degree image.
 8. The method of claim 1, wherein the corrected tilt forms an angle of approximately 90 degrees with the ground.
 9. An FOV correction method performed in a server implemented with a computer, the method comprising: receiving a protocol or a message for requesting a streaming service for a 360-degree image from a user terminal; analyzing application information included in the protocol or the message for requesting the streaming service, and determining a content consumption environment of the user terminal, said determining comprising determining whether the content consumption environment is a smartphone environment; if the content consumption environment is a smartphone environment, connecting configuration information, to support correction for a tilt of a mobile terminal; transmitting the 360-degree image connected to the configuration information to the mobile terminal, the 360-degree image being transmitted as a response to the protocol or the message for requesting the streaming service; wherein the mobile terminal plays the 360-degree image based on the correction of the tilt, wherein the FOV played by the mobile terminal is different than an FOV that would have been played without the correction for the tilt, and wherein the FOV played by the mobile terminal includes a scene of the 360-degree image designated as important.
 10. The method of claim 9, wherein the tilt is corrected to form an angle of approximately 90 degrees with the ground.
 11. The method of claim 9, wherein the configuration information is connected if and only if the content consumption environment is a smartphone environment.
 12. The method of claim 9, wherein an initial FOV of the 360-degree image connected to the configuration information is different than if the 360-degree image were not connected to the configuration information.
 13. A computer program stored in a non-transitory computer-readable medium to execute an FOV correction method performed in a mobile terminal implemented with a computer, the method comprising: requesting to load a 360-degree image; correcting for a tilt of the mobile terminal, for determining an FOV of the 360-degree image; and playing the 360-degree image by exposing the FOV depending on at least said correcting, wherein the FOV is different than an FOV that would be exposed without said correcting, and wherein the FOV includes a scene of the 360-degree image designated as important.
 14. A mobile terminal implemented with a computer, the mobile terminal comprising: at least one processor configured to execute a computer-readable instruction, wherein the at least one processor is configured to: request to load a 360-degree image; correct for a tilt of the mobile terminal, for determining an FOV of the 360-degree image; and play the 360-degree image by exposing the FOV depending on at least said correcting, wherein the FOV is different than an FOV that would be exposed without said correcting, and wherein the FOV includes a scene of the 360-degree image designated as important.
 15. A server implemented with a computer, the server comprising: at least one processor configured to execute a computer-readable instruction, wherein the at least one processor is configured to: receive a protocol or a message for requesting a streaming service for a 360-degree image from a user terminal; analyze application information included in the protocol or the message for requesting the streaming service, and determine a content consumption environment of the user terminal, including at least determining whether the content consumption environment is a smartphone environment; if the content consumption environment is a smartphone environment, connect configuration information, to support the correction for a tilt of a mobile terminal such that an initial FOV of the 360-degree image is different than an FOV that would be played without the correction for the tilt and such that the initial FOV includes a scene of the 360-degree image designated as important; and transmit the 360-degree image connected to the configuration information to the mobile terminal, the 360-degree image being transmitted as a response to the protocol or the message for requesting the streaming service.
 16. The server of claim 15, wherein the process is configured to connect said configuration information if and only if the content consumption environment is a smartphone environment. 